The Internet is a vast and expanding network of networks of computers and other devices linked together by various telecommunications media, enabling all these computers and other devices to exchange and share data. Sites on the Internet provide information about a myriad of corporations and products, as well as educational, research and entertainment information and services. An estimated 30 million people worldwide use the Internet today, with 100 million predicted to be on the "net" in a matter of years.
A computer or resource that is attached to the Internet is often referred to as a "host." Examples of such resources include conventional computer systems that are made up of one or more processors, associated memory (typically volatile and non-volatile) and other storage devices and peripherals that allow for connection to the Internet or other networks (e.g., modems, network interfaces and the like). In most cases, the hosting resource may be embodied as hardware and/or software components of a server or other computer system that includes an interface, which allows for some dialog with users thereof. Generally, such a server will be accessed through the Internet (e.g., via Web browsers such as Netscape's Navigator.TM. and Communicator.TM. and Microsoft's Internet Explorer.TM.) in the conventional fashion.
Briefly, if an Internet user desires to establish a connection with a host (e.g., to view a Web page located thereat), the user might enter into a Web browser program the URL (or Web address) corresponding to that host. One example of such a URL is "http://www.domain.com". In this example, the first element of the URL is a transfer protocol (most commonly, "http" standing for hypertext transfer protocol, but others include "mailto" for electronic mail, "ftp" for file transfer protocol, and "nntp" for network news transfer protocol). The remaining elements of this URL (in this case, "www" standing for World Wide Web--the Internet's graphical user interface--and "domain.com") are an alias for the "fully qualified domain name" of the host.
Each fully qualified domain name, in its most generic form, includes three elements. Taking "computer.host.com" as an example, the three elements are the hostname ("computer"), a domain name ("host") and a top-level domain ("com"). Further, each fully qualified domain name is unique throughout the Internet and corresponds to a numerical Internet protocol (IP) address. IP addresses facilitate communications between hosts and clients in the same way that physical addresses (e.g., 123 Main Street, Anytown, Anycity) facilitate correspondence by mail. Each IP address is made up of four groups of numbers separated by decimals. Thus, in the case of the hypothetical host "computer.domain.com", the corresponding IP address might be 123.456.78.91. A given host looks up the IP addresses of other hosts on the Internet through a system known as domain name service.
Thus, once a URL is entered into a browser, the corresponding IP address is looked up in a process facilitated by a top-level server. In other words, all queries for addresses are routed to certain computers, the so-called top-level servers. The top-level server matches the domain name to an IP address of a domain name server capable of directing the inquiry to the computer hosting the sought after Web page (or other content) by matching an alphanumeric name such as www.domain.com with its numeric IP address.
In addition to Web pages and the like, more and more Internet users are accessing multimedia content (e.g., files that include high quality graphical images, movies and/or sound). This creates difficulties because such files are usually quite large while the bandwidth available through the Internet is limited. Thus, in order to make multimedia files usable, streaming is often employed.
With conventional files (e.g., data files), clients (e.g., Web browsers) completely download the requested content before viewing it. This technique works well for relatively small files, but often suffers from unacceptable (from the point of view of the user) delays when large multimedia files are involved. Streaming is the term given to a technique wherein a client downloads a portion of a file, decompresses (if necessary) that portion, and starts playing the contents thereof (e.g., audio and/or video) before the rest of the file arrives. A buffer of information is built up before playback starts, so as to prevent underflows if the remaining data is delayed during transmission. Furthermore, subsequent portions of the multimedia file are downloaded during playback to keep the buffer relatively full. This technique thus accommodates the downloading and playing of large multimedia files without incurring lengthy delays before the content is available for viewing.
Multimedia files are often transported over the Internet using special transport protocols. For example, the real-time transport protocol (RTP) provides delivery service for multimedia applications and also provides means for multimedia applications to work over networks. RTP does not, however, provide guaranteed or in-sequence delivery (and hence it is referred to as an unreliable transport protocol), but does provide a packet sequence number that can be used to detect missing packets and to reconstruct an original transmission sequence.
RTP usually carries data in the form of packets, using the user datagram protocol (UDP) as the delivery mechanism. UDP provides a "wrapper" around data packets, with the wrapper providing for multiplexing and demultiplexing as well as error checking services. Essentially, a UDP packet is made up of a UDP header and UDP data encapsulated as the data portion of an IP packet. The IP packet itself includes an IP header (which includes the address information discussed above) as well as the user data (i.e. the multimedia content of interest) as a payload.
In some cases, RTP is used with other protocols, such as the transmission control protocol (TCP). Unlike UDP, TCP provides a reliable, error-free, full-duplex channel between two computers. TCP uses IP to transfer data, but provides mechanisms to take care of lost or duplicated IP datagrams (i.e., packets) and to ensure proper sequencing thereof. Thus, TCP provides reliable end-to-end transport, ensuring that what is received is an exact duplicate of what is transmitted.
Where broadcasts of "live" information are concerned, RTP usually uses UDP as the delivery transport. This is because TCP may introduce unacceptable delays in receipt. Further, when an application starts an RTP session, a second port for communication according to the real time control protocol (RTCP) is opened. RTCP works in conjunction with RTP to provide flow control and congestion control services. The idea is that the exchange of RTCP packets between a client and server can be used to adjust the rate of transmission of the RTP packets, etc.
Associated with RTP is the real time streaming protocol (RTSP). RTSP is a client-server multimedia presentation control protocol that control functionality such as content type, interactive stream control, error mitigation, bandwidth negotiation, multicast, live broadcasting and monitoring. Just as HTTP transports HTML (hypertext markup language--an instruction set that allows an HTTP client to render a desired image, etc.), RTSP handles data. The difference is that while HTTP clients always make requests and HTTP servers always service those requests, RTSP is bi-directional, with both servers and clients making requests and servicing them. RTSP accomplishes data transfer using TCP or UDP.
Thus, RTSP is a generally a TCP connection over which commands are sent and responses are returned. Clients negotiate data channels with servers via SETUP commands. These channels typically specify that data will be sent as RTP/RTCP "packets", but the transport type may be specified and/or modified as part of the negotiation. Further, the negotiations may determine whether the packets will be sent over TCP (e.g., as binary packet data imbedded in an RTSP command stream via an escape sequence) or UDP (e.g., as true packets).
The RTP portion of a channel contains actual media data for single stream flows (e.g., compressed audio data). In contrast, an RTCP portion of a channel (which typically is assigned one UDP port number or TCP channel number larger than the RTP port number or channel--for example, UDP port 6970 for RTP and 6971 for RTCP) usually contains clock-synchronization data and client-server control/status messages. As indicated above, RTP data typically flows in one direction, from the server to the client. RTCP packets are typically sent in both directions, for example as client status report messages and server status report messages.
The following dialog illustrates (from a client's point of view) a portion of an RTSP session wherein a media description is retrieved and the single audio stream specified thereby is played. Those of ordinary skill in the art will recognize that this is a "live" stream as opposed to prerecorded content, as is evident from the absence of a "range" tag in the session description protocol (SDP). In the initial client-to-server communication, a request to describe the stream located at a particular Web address (rtsp://qt.macroradio.net/gogaga) is sent, along with suggested connection parameters, such as a bandwidth:
Send data (130 bytes).
&lt;00000000&lt;DESCRIBE rtsp://qt.macroradio.net/gogaga RTSP/1.0
&lt;00000033&lt;CSeq: 1
&lt;0000003C&lt;Accept: application/sdp
&lt;00000055&lt;Bandwidth: 112000
&lt;00000068&lt;User-Agent: QTS/1.0b22
&lt;00000080&lt;
The server responds with a description of the stream as part of an SDP:
Receive data (566 bytes).
&gt;00000000&gt;RTSP/1.0 200 OK
&gt;00000011&gt;Server: QTSS/v65
&gt;00000023&gt;Cseq: 1
&gt;0000002C&gt;Content-Type: application/sdp
&gt;0000004B&gt;Content-Base: rtsp://qt.macroradio.net/gogaga/
&gt;0000007B&gt;Content-length: 420
&gt;00000090&gt;
&gt;00000092&gt;v=0
&gt;00000097&gt;o=3134316689 3134906918 IN IP4 192.231.139.83
&gt;000000C7&gt;s=GoGaGa Brand Radio
&gt;000000DD&gt;i=&lt;&lt;A9&gt;&gt;1999--All rights reserved -
&gt;000000FF&gt;u=http://www.macroradio.net
&gt;0000011C&gt;a=x-qt-text-nam:GoGaGa Brand Radio
&gt;00000140&gt;a=x-qt-text-cpy:&lt;&lt;A9&gt;&gt;1999--All rights reserved
&gt;0000016D&gt;a=x-qt-text-cpy:(c) 1999 ERC: The Eclectic Radio Company,
&gt;000001A7&gt;LLC.
&gt;000001AD&gt;t=3134316689 3134320289
&gt;000001C6&gt;c=IN IP4 0.0.0.0
&gt;000001D8&gt;a=control:*
&gt;000001E5&gt;m=audio 0 RTP/AVP 97
&gt;000001FB&gt;a=rtpmap:97 X-QT
&gt;0000020D&gt;a=x-bufferdelay:10
&gt;00000221&gt;a=control:trackID=1
The client then defines conditions for a playback, using a specified port:
Send data (143 bytes).
&lt;00000082&lt;SETUP rtsp://qt.macroradio.net/gogaga/tracklD=1 RTSP/1.0
&lt;000000BC&lt;CSeq: 2
&lt;000000C5&lt;Transport: RTP/AVP;unicast;client.sub.- port=6970-6971
&lt;000000F7&lt;User-Agent: QTS/1.0b22
&lt;0000010F&lt;
The server responds with the port address from which it will play:
Receive data (165 bytes).
&gt;00000236&gt;RTSP/1.0 200 OK
&gt;00000247&gt;Server: QTSS/v65
&gt;00000259&gt;Cseq: 2
&gt;00000262&gt;Session: 1411920655;timeout=60
&gt;00000282&gt;Transport: rtp/avp;source=192.231.139.182;server.sub.- port=2000-2001;
&gt;000002C2&gt;client.sub.- port=6970-6971
&gt;000002D9&gt;
The client then initiates play back using the specified port addresses:
Send data (125 bytes).
&lt;00000111&lt;PLAY rtsp://qt.macroradio.net/gogaga RTSP/1.0
&lt;00000140&lt;CSeq:3
&lt;00000149&lt;Range: npt=0.000000--
&lt;0000015F&lt;Session: 1411920655
&lt;00000174&lt;User-Agent: QTS/1.0b22
&lt;0000018C&lt;
In response, the server begins playing the selected stream:
Receive data (92 bytes).
&gt;000002DB&gt;RTSP/1.0 200 OK
&gt;000002EC&gt;Server: QTSS/v65
&gt;000002FE&gt;Cseq: 3
&gt;00000307&gt;Session: 1411920655
&gt;0000031C&gt;RTP-Info: url=trackID=1
&gt;00000335&gt;
At this point, RTP media data and RTCP control packets will start flowing between the specified UDP ports, i.e., RTP data from server port 2000 to client port 6970, and RTCP packets between server port 2001 and client port 6971. Note that the server's address 192.231.139.182 can be seen in the SETUP response above. An example of an RTCP packet transmitted from the client (port 6971) to the server (port 2001) is shown below:
Send packet data to port 1 (84 bytes). &lt;00000000&lt; 80 C9 00 01 00 00 62 76 81 CA 00 12 00 00 62 76 ......bv......bv &lt;00000010&lt; 01 0D 51 54 53 20 38 30 35 32 31 38 36 39 33 02 ..QTS 805218693. &lt;00000020&lt; 13 51 75 69 63 6B 54 69 6D 65 20 53 74 72 65 61 .QuickTime Strea &lt;00000030&lt; 6D 69 6E 67 06 1A 51 75 69 63 6B 54 69 6D 65 20 ming..QuickTime &lt;00000040&lt; 53 74 72 65 61 6D 69 6E 67 2F 31 2E 30 62 32 32 Streaming/1.0b22 &lt;00000050&lt; 00 00 00 D2 ....
If one were to decode the data, it may translate to "client received 5% loss and 123456 bytes and client "name" is 'QuickTime Streaming' client".
An RTCP packet sent from the server (port 2001) to the client (port 6971) may resemble the following:
 Receive packet data from 192.231.139.182:2001 (108 bytes). &gt;00000000&gt; 80 C8 00 06 00 00 45 38 BA F0 68 00 42 1F 8E 44 ......E8..h.B..D &gt;00000010&gt; 3D 25 45 EB 00 E7 A3 3F BB 8C 3A 78 81 CA 00 13 =%E....?..:x.... &gt;00000020&gt; 00 00 45 38 01 18 51 54 53 40 6C 6F 63 61 6C 68 ..E8..QTS@localh &gt;00000030&gt; 6F 73 74 20 31 35 36 31 36 31 35 34 36 39 02 13 ost 1561615469.. &gt;00000040&gt; 51 75 69 63 6B 54 69 6D 65 20 53 74 72 65 61 6D QuickTime Stream &gt;00000050&gt; 69 6E 67 06 13 51 75 69 63 6B 54 69 6D 65 20 53 ing..QuickTime S &gt;00000060&gt; 74 72 65 61 6D 69 6E 67 00 FF 00 40 treaming...@
Once decoded, this information may state, "RTP time t means universal (wall clock) time y and server "name" is "QuickTime Streaming server". This is essentially a reference to an absolute reference time source used by the server.
The actual RTP media packets transmitted from the server (port 2000) to the client (port 6970) may resemble the following:
 Receive packet data from 192.231.139.182:2000 (200 bytes). &gt;00000000&gt; 80 E1 BC 73 3D 21 F8 1B 00 00 45 38 14 00 80 01 ...s=!....E8.... &gt;00000010&gt; 02 B6 4B 19 09 14 80 9E 5D 26 35 24 88 64 2A 20 ..K....]&5$.d* &gt;00000020&gt; D0 C2 98 16 92 55 41 9C 82 46 16 35 9D A8 D7 27 ....UA..F.5...' &gt;00000030&gt; 13 1A B7 37 D6 E4 05 5B 40 AF E7 11 D3 84 9C B8 ...7...[@...... &gt;00000040&gt; 45 8D 51 01 F1 A4 C5 97 0B 58 88 2A 4A D1 C4 13 E.Q......X.*J... &gt;00000050&gt; FC 8C 58 A5 46 8A A2 3B 63 66 6F 23 2F 38 1B 61 ..X.F..;cfo#/8.a &gt;00000060&gt; 0B 15 2A D3 49 22 C9 98 C8 0F 16 40 1A 53 9D A8 ..*.I".....@.S.. &gt;00000070&gt; 79 F1 CE EE C6 19 B1 26 C5 A8 CB 4D 4B 3B F3 73 y......&...MK;.s &gt;00000080&gt; 4C 6A 33 5F D3 5F 2C 46 60 84 C0 08 14 14 26 EC Lj3_._,F.....&. &gt;00000090&gt; 5E DF 49 49 48 B5 B3 02 5F 88 F5 EC 29 10 AB 72 .IIH..._...)..r &gt;000000A0&gt; A6 D8 3E D4 9A D2 14 2A 6F 86 AD 22 9E 0B 4C 50 ..&gt;....*o.."..LP &gt;000000B0&gt; 5C BC 0B 88 6D 13 0C 34 3C 44 CB 92 BB 6B 1B 18 .backslash....m..4&lt;D...k.. &gt;000000C0&gt; 51 1C 7D 12 01 00 00 00 Q.}.....
Finally, upon conclusion of the playback or at some other point, the client may decide to quit, so an instruction is passed to server over RTSP (the TCP connection is still open during the playback) to stop everything on this "session":
Send data (107 bytes).
&lt;0000018E&lt;TEARDOWN rtsp://qt.macroradio.net/gogaga RTSP/1.0
&lt;000001C1&lt;CSeq: 4
&lt;000001CA&lt;Session: 1411920655
&lt;000001DF&lt;User-Agent: QTS/1.0b22
&lt;000001F7&lt;
Although these and other transport protocols exist, there still exist problems with the viewing of streaming content over public networks or networks of networks such as the Internet. For example, whenever unreliable transport protocols (e.g., RTP) are used, there can be significant data loss between the content source (e.g., the server) and the content consumer (i.e., the client), depending on the network traffic conditions. If this loss is high (e.g., 10% or more), the viewing quality can be degraded to the point where it is unacceptable to a user.
Others have attempted to solve this problem (improved viewing quality or, more generally, improved user experience in the face of data loss), but have primarily concentrated on trying to make the transport of information from source to client more reliable. Thus, for example, others have attempted to use TCP rather than UDP as the transmission protocol. Although TCP guarantees that all packets that make up a file will arrive and will be in sequence when ultimately played out, it does so by requesting retransmissions of any lost packets. This not only introduces delay into any playback (e.g., while waiting for lost packets to be retransmitted), it also adds to the total volume of network traffic thus leading to still further packet loss due to congestion and the like. The delays introduced by the use of TCP often mean that the ultimate user experience is poor, and perhaps even less acceptable than would be the case if lost packets were simply overlooked.
Other solutions have proposed using network bandwidth reservation protocols in place of unreliable transmission protocols. Unfortunately, such protocols are not always available end-to-end, and so this solution is not always an option.
Another strategy for dealing with the loss of data between the source and the requesting client is to control the amount of data being transmitted. Where possible, clients that are experiencing significant data loss may instruct the server to send less data. Thus, in the scenario of a streaming movie, the server may be instructed to send only key frames and not any (or perhaps just a few) difference frames. However, this strategy is only successful where the data loss is due to actual bandwidth overloading at the client and not due to other factors in the intervening network. For example, if the packet loss is due to consistent buffer overflow at the receiver, instructing the server to send less data may prevent such overflows and provide a better user experience. Where, however, packet loss is due to overall network congestion, instructing the server to send less data will have no effect on the packet loss, because the insignificant reduction in the total number of packets within the network due to the stream under consideration will not be sufficient to dramatically affect the packet loss rate for that stream. The net result will simply be roughly the same packet loss rate over a fewer number of transmitted packets--leading to an even worse condition.
What is needed, therefore, is a new solution to the problem of dealing with data loss during the broadcast of live, streaming content.